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The invention relates to a method and a device for sensing the spatial position of an object. 

Such method is required, in particular, for what is known as HMD devices (Head Mounted 
Display devices), using which a viewer wears a display device on his head, said device 
generating images that he can perceive. For this purpose, the HMD device may be provided 
such that the viewer sees the generated image only, or that he perceives the generated image 
superimposed on the surroundings (so-called augmented reality). In particular, augmented 
reality requires that the viewer's movements and/or the position of his head be sensed 
continuously and be considered in generating the image, so as to realize said superposition in 
the best possible way. 

For this purpose, magnetic methods are often used, which require exact measurement of a 
magnetic field (statically or dynamically). Magnetic methods have the disadvantage that any 
(ferro)metallic objects located in the area of observation influence the magnetic field and thus 
contribute to reduced precision. In particular, if said metallic objects are not statically located in 
one location, renewed calibration is also required in addition. 

Furthermore, there are ultrasonic systems, which only have a limited resolution, however, due 
to the wavelength of use. Further, they are also highly susceptible to interference. 

Further, there are systems comprising gyroscopes, but these are particularly complex. 

In view thereof, it is an object of the invention to propose a method and a device for sensing the 
spatial position of an object, which require reduced complexity and exhibit high precision. 

According to the invention, the object is achieved by a method of sensing the spatial position of 
an object, which comprises a mounting step in which three light sources are mounted on the 
object such that they define a triangle, an activating step in which the light sources are turned 
on, a recording step in which the object is recorded simultaneously from first and second 
positions, with light sources turned on, as well as an evaluating step in which the positions of 
the light sources in the recorded images are determined and the position of the object is 
computed on the basis of the determined positions of the light sources. 
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By simultaneously recording the object with turned-on light sources, all three of them being 
turned-on at the same time while recording, a pair of stereo images of the triangle defined by 
the light sources is generated, using which the spatial position (location and orientation) of the 
triangle can be determined in a known manner, in order to also determine the spatial position of 
5 the object therefrom. 

The object may be, for example, a helmet or any other support for a display device of an HMD 
device which a viewer can mount on his head, said head-mounted condition enabling him to 
perceive images generated by means of the display device. Said support may also be provided, 
10 for example, in the manner of spectacles comprising two bows for both ears, and in the present 
case, three light sources defining a triangle may be provided at both, or only one, of said bows. 

In particular, the method of the invention allows the images of the light sources to be separated 
from the remaining image background in order to determine the positions of the light sources. 
15 This provides a stereo pair of images having three isolated points (images of the light sources), 
which can be evaluated with particular ease. 

The separation of the images of the light sources from the remaining image background may be 
carried out, for example, electronically. This enables electronic image processing wherein said 
20 separation is effected via the brightness or color of the recorded (turned-on) light sources. 

Particularly preferably, the light sources are turned off prior to the evaluating step and the object 
is recorded, with the light sources turned off, from the first and second positions at the same 
time, with the image recorded while the light sources were turned off being subtracted from the 

25 images recorded while the light sources were turned on in the evaluating step, in order to 
determine the positions of the light sources for each recording position. In doing so, the entire 
image background is separated from the images of the light sources, so that a stereo pair of 
images comprising only three isolated points (images of the light sources) is present. Said 
subtraction of the images from each other is easily carried out using conventional image 

30 processing programs 

A preferred embodiment of the method according to the invention consists in that, in the 
mounting step, more than three light sources are mounted on the object, of which only three 
light sources are turned on each time in the activating step. It can thus be ensured that three 
35 light sources can always be recorded at the same time from both positions, even during more 
extensive movements of the object. 
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ln particular, if more than three light sources are mounted on the object, it is possible to always 
turn on those three light sources in the activating step which form the largest triangle that can 
be recorded from both positions. The largest triangle can be that triangle which has the largest 



5 position. 

A particularly preferred embodiment of the method according to the invention consists in that 
the light sources are operated in a pulsed manner. In this case, it is particularly easy to always 
generate one image pair with turned-on light sources and one image pair with turned-off light 
10 sources, which image pairs can then be used to separate the image background from the 
images of the light sources. 

Thus, images can bo recorded synchronously using the light sources operated in a pulsed 
manner so as to record an image pair with turned-on light sources and an image pair with 
15 turned-off light sources in an alternating manner. Two consecutive image pairs (i.e. one image 
pair with turned-on light sources and one image pair with turned-off light sources) can then be 
used for the above-described separation of the image background (e.g. by suitable subtraction 
of the images). 

20 Further, the light sources can be turned on and recorded individually, one after the other, in 
order to identify the light sources. This makes it particularly easy to identify the individual light 
sources in the recorded images by means of three tumed-on light sources, thus ensuring that 
the position of the object is correctly determined. 

25 In the method according to the invention, the light sources can be controlled in a wireless 
manner. This is advantageous, in particular, when applying the method of the invention to HMD 
devices, because undesired cable connections can be omitted. 

A further embodiment of the method according to the invention consists in that the light sources, 
30 in the turned-on condition, emit light (or electromagnetic radiat ; on, respectively) in the non- 
visible wavelength range (e.g. in the infrared range). This leads to the advantage that there is 
no blinking or illumination, respectively, of the light sources disturbing the viewer (e.g. in the 
case of an HMD device) carrying out said method. 

35 The object is also achieved by a device for sensing the spatial position of an object, wherein the 
device comprises ttree light sources, which can be mounted on the object, two spaced apart 
image-recording devices, whose image-recording fields overlap, a control device which causes 
the light sources to be turned on and causes simultaneous recording of the object wij|j tamed*- 



area or the longest circumference. This increases the precision achievable in determining the 
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on light sources by both image-recording devices, as well as an evaluating unit, which 
determines the position of the light sources in the recorded images and the position of the 
object on the basis of the determined positions of the light sources. 

5 Using this device, the positions of the light sources and thus also the spatial position of the 
object can be easily determined by determining the positions of the light sources in the recorded 
images. 

The control device and the evaluating unit can be realized by means of a conventional computer 
10 with suitable software. 

An advantageous further embodiment of the device according to the invention consists in that 
the evaluating unit separates the images of the light sources from the remaining image 
background in order to determine the positions of the light sources. Such separation can be 
15 effected, in particular, electronically. 

Thus, for example, the control device can have the effect that the light sources are turned off 
and, at the same time, the object is recorded with turned-off light sources by both image- 
recording devices, and, for each image-recording device, the evaluating unit can subtract the 
20 recorded image with turned-off light sources from the recorded image with turned-on light 
sources when determining the positions of the light sources. This is easy to do using 
conventional image-processing programs and leads to a stereo pair of images comprising three 
isolated points (images of the light sources). The position of the object is particularly easy to 
determine from said stereo light pair. 

25 

A preferred embodiment uses infrared diodes (IR diodes) as light sources on the object (e.g. on 
. the helmet) and infrared-sensitive image pick-ups or image-recording devices, respectively. An 
infrared filter, through which only light having a wavelength of more than 830 nm can pass, is 
placed in front of each image pick-up. This results in a considerably darker representation of the 
30 typical surrounding light and, thus, of the recorded scene. However, since the wavelength of the 
IR diodes is 880 nm, the light of the IR diodes arrives at the image pick-up without hindrance as 
a strong useful signal. This measure clearly increases the ratio of useful signal to interference 
signal when subtracting the images, which also shows in a substantially improved contrast of 
the points isolated in the image. 

35 

A further specific embodiment of the invention makes use of the evaluation of the isolated points 
by means of a further image-analyzing measure, by analyzing the ensuing luminous spot 
(image of the light source in the recorded image) in relation to a center of gravitation. This 
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results in a further substantial increase in the precision of the device according to the invention 
in position-determining. 

Since the geometry of the light sources (e.g. light emitting diodes) has a direct influence on 
5 computing the center of gravity of the luminous spot, there is a further improvement according 
to the invention. Fo=' this purpose, the light sources (e.g. light emitting diodes) are buried in a 
cylindrical bore of a material (e.g. aluminum, for IR diodes) that is not transparent in the 
wavelength range of the light or of the radiation, respectively, from the light sources (e.g. in the 
infrared range in the case of IR diodes as light sources). The remaining circular, open surface of 
10 the bore is covered by a thin diffusor plate, for example, which has a wide scattering angle of 
e.g. 60°. This arrangement makes the luminous spot always appear as a circular or oval shape 
and the center of gravity is accordingly formed in a more precise manner. 

At great distances of the light sources (e.g. light emitting diodes or IR diodes) from the image 
15 pick-up, these can be operated with a correspondingly stronger current, so that optimal 
contrasts are always achieved even at different distances. Thus, the brightness of the light 
sources (e.g. of the IR diodes) is controlled by a control circuit, which may be part of the control 
main circuit, in such a manner that a stronger current is imposed as the distance from the image 
pick-up or from the image-recording devices, respectively, increases. With reference to said 
20 distance, this may also be effected, for example, gradually with a certain hysteresis. 

Further, in the device according to the invention, a control unit may be provided, which is 
connected with the light sources and controls the light sources on the basis of signals wirelessly 
transmitted by the device, wherein, in particular, a current or voltage source, respectively, is 
25 also provided for the light sources. This readily allows a portable design of the light-source 
arrangement together with the control unit, which is an advantage, in particular, in HMD 
devices. 

A particularly preferred embodiment of the method according to the invention consists in that 
30 light emitting diodes, in particular infrared-light emitting diodes, are employed as light sources. 
Light emitting diodes are small and inexpensive light sources which, in addition, have a very 
long service life, thus allowing to ensure the reliability of the device according to the invention. 
The use of infrared-light emitting diodes also has the advantage that the emitted infrared 
radiation is not perceivable by a viewer and thus also not disturbing. 



Further, in the device according to the invention, the object may comprise a display unit that can 
be mounted on a viewer's head and, in the mounted condition, can generate an image which is 
perceivable by the viewer. The display unit may be provided such that the observer can 
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perceive the generated image only, or the generated image superimposed on the environment. 
In this way, an HMD device can be realized which allows easy and reliable determination of the 
position of the viewer's head. 

In the device according to the invention, the light sources can be connected with the object in a 
fixed manner and define a triangle. This ensures that the position of the object can always be 
deduced from the position of the light sources (or the triangle of light sources, respectively). 

In a further embodiment of the method according to the invention, it is possible that the light 
sources comprise a predetermined emission spectrum and the recording devices only record 
light having said predetermined emission spectrum. Thus, a separation of the image of the light 
sources from the image background is realized already during recording. This is possible, in 
particular, by the use of corresponding filters which are provided for each of said recording 
means and only allow the passage of light of the predetermined emission spectrum. 

The invention is explained in more detail below, essentially by way of example, with reference to 
the drawings, wherein: 

Fig. 1 schematically shows an embodiment of the device according to the invention in 



Fig. 2 schematically shows a simplified operational diagram of the device of Fig. 1 , and 

Fig. 3 schematically shows a further embodiment of the device according to the invention in an 



The device according to the invention for sensing the spatial position of an object comprises a 
helmet 1 having three infrared-light emitting diodes 2, 3, and 4 mounted thereon, of which a first 
light emitting diode 2 is arranged on top of the helmet and the other two light emitting diodes 3 
and 4 are arranged at the lower edge 7 of the helmet 1 with an offset of about 90° in the 
peripheral direction between them. 

Further, a control unit 8 for the light emitting diodes 2 to 4 is provided on the helmet 1, which 
control unit will be described in more detail later. 

A display unit 9 is mounted on the helmet 1 in such a manner that a viewer (not shown) wearing 
the helmet 1 can perceive the images generated by means of the display unit 9. The helmet 1 
comprising the display unit 9 is a so-called HMD. „ e ~ v 0u* 6c - 
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The device according to the invention further comprises two cameras 10, 11, which can record 
the helmet 1 or the light emitting diodes 2 to 4, respectively, from different positions. For this 
purpose, the cameras 10, 11 are arranged and designed such that their image-recording areas 
5 12, 13 overlap at least partially and the overlapping region of both image-recording areas 12 
and 13 covers the range of movement of the helmet 1 during its intended use. 

The cameras 10 and 1 1 are controlled by means of a control device 14 as are the light emitting 
diodes 2 to 4. 

10 

As is evident, in particular, from Fig. 2, the control device 14 includes, for each of the cameras 
10, 11, a digitizing unit 15, 16, which is controlled by a control module 17 of the control device 
14. 

15 Further, the control device 14 comprises a transmitter unit 18, via which the control module 17 
wirelessly transmits control signals for the light diodes 2 to 4 to the control unit 8 (as indicated 
by the arrow A), which signals are received by a receiving unit 19 of the control unit 8. 

In addition to the receiving unit 20, the control unit 8 comprises n control unit, which connects 
20 the light emitting diodes 2 to 4 with a voltage supply 21 of the control unit 8 as a function of the 
signals received. 

The digitizing units 15 and 16 are connected with an evaluating unit 22, which uses the 
(digitized) images to compute the positions of the light emitting diodes 2 to 4 and then computes 
25 therefrom the position of the helmet 1 and thus the position of the viewer's head. The data 
relating to the position of the helmet 1 are supplied, by the evaluating unit 22, to the control 
module 17 as well as to a display control 23 of the display unit 9, so that, for example, the 
image generated by means of the display unit 9 can be positioned correctly in the environment 
that is still perceivable through the display unit 9. 

30 

The control device 14 and the evaluating unit 22 as well as the display control 23 can be 
realized by means of one or more computers with suitable software. 

In operation, the light emitting diodes 2 to 4 are operated in a pulsed manner and are always 
35 turned on and off at the same time. In the embodiment shown in Fig. 1 , the light emitting diodes 
2 to 4 define the indicated triangle 24. 
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When the three light emitting diodes 2, 3 and 4 are turned on, an image of the helmet 1 is taken 
by both cameras 10 and 11, at the same time, with the turned-on light emitting diodes 2 to 4. 
The recorded images are digitized by means of the digitizing units 15 and 16 and stored in the 
evaluating unit 22. 



The light emitting diodes 2 to 4 are turned off thereafter (by the pulsed operation) and the 
cameras 10 and 11 simultaneously record the helmet 1 with turned-off light emitting diodes 2 to 
4. These images are again digitized as well by means of the digitizing units 15 and 16 and 
supplied to the evaluating unit 22. 



The evaluating unit 22 then subtracts the image of the helmet taken by camera 10 with turned- 
off light emitting diodes 2 to 4 from the image of the helmet taken by camera 10 with turned-on 
light emitting diodes 2 to 4. Both images taken by camera 1 1 are subtracted from one another in 
the same manner, so that a stereo pair of images with, ideally, only three isolated points (the 

15 images of the light emitting diodes 2 to 4) is present, from which the spatial position of the 
triangle 24 defined by the light emitting diodes 2 to 4 can be computed by known methods 
taking into consideration the positions of both cameras 10 and 11. Since the light emitting 
diodes 2 to 4 are mounted on the helmet 1 and their exact positions on the helmet are known 
(for example, by means of a one-time calibration), the position (location and orientation) of the 

20 helmet 1 can be precisely determined. 

Depending on the spatial position of the helmet 1 and thus also on the spatial position of the 
display unit 9, the display unit 9 is controlled as desired by menns of the display control 23. 
Thus, for example, the (stereo) image generated by means cf the display unit 9 can be 
25 generated such that it always appears at the same spatial location, independent of the 
movements of the viewer. 

The light emitting diodes 2 to 4 are subsequently turned on again and are recorded, at the same 
time, by the cameras 10 and 11, and, in a subsequent turned-off state of the light emitting 
30 diodes, the helmet 1 is in turn recorded by both cameras 10 and 11. A stereo pair of images 
with three isolated points is in turn generated from these recorded images in the same manner 
as above. 

Thus, the spatial position of the helmet 1 and, consequently, the spatial orientation of the 
35 viewer's head can be continuously determined. 
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It is also possible, of course, to first record the helmet 1 with turned-off light emitting diodes 2 to 
4 and then the helmet 1 with tumed-on light emitting diodes 2 to 4 and, in turn, to generate 
therefrom the stereo pair of images with the isolated points. 

5 A further embodiment of the device according to the invention is shown in Fig. 3, which differs 
from that of Fig. 1 substantially only in that two further light emitting diodes 5 and 6 are arranged 
at the lower edge 7 of the helmet 1 such that they are also located in the overlapping region of 
the two image-recording areas 12 and 13 of the cameras 10 and 11. Identical elements of the 
embodiments shown in Fig. 1 and 3 are identified by the same reference numerals and their 
10 description is not repeated. 

Since four light emitting diodes 3 to 6 are provided at the lower edge 7 of the helmet 1 in the 
embodiment of Fig. 3, several different triangles can be generated in that the first light emitting 
diode 2 is always turned on in combination with two of the light emitting diodes 3 to 6 arranged 
1 5 at the lower edge 7 of the helmet 1 . 

The purpose of this may be, on the one hand, to illuminate the diode triangle by which the 
greatest precision can be achieved in the evaluation. This may be, for example, the diode 
triangle having the largest area or the longest circumference. 

20 

Alternatively, the diodes 2 to 6 can also be activated so as to create the impression of a diode 
triangle moving around the periphery of the helmet 1. For example, first the diodes 2, 4 and 5, 
then the diodes 2, 5 and 3, and then the diodes 2, 3 and 6 are activated. Then, the diode 
triangle 2, 4 and 5 is started with again. This generates a partial circulation of the diode triangle 
25 on the helmet 1. Such a circulatory method of activation of the diode triangles is particularly 
suitable, if a prediction of the movement of the helmet 1 is to be computed also by means of the 
evaluating unit 22. 

Further light emitting diodes (not shown) at the lower edge 7 of the helmet 1 as well as further 
30 cameras (not shown) may be provided such that each triangle of light emitting diodes containing 
the light emitting diode 2 can be recorded by at least two different cameras at the same time. In 
this case, a diode triangle moving around the periphery of the helmet 1 can be generated which 
can always be evaluated with respect to the spatial position of the helmet 1, because it is 
always recorded by two cameras at the same time. 

35 

Furthermore, a synchronization of the evaluating unit 21 with the individual diodes 2 to 6 may be 
effected such that the diodes 2 to 6 are individually turned on and off, one after the other. Such 
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synchronization can be effected at the beginning of the process, at regular time intervals or as 
required. 

Instead of the cameras 10 and 11 as shown and the associated digitizing units 15 and 16, 
5 digital cameras may also be used, of course, so that the digitizing units 14, 15 and 16 no longer 
have to be provided separately. 
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